To realize automatic and quantitative assessment in home-based upper extremity rehabilitation for stroke, an Extreme Learning Machine (ELM) based prediction model was proposed to automatically estimate the Fugl-Meyer Assessment (FMA) scale score for shoulder-elbow section. Two accelerometers were utilized for data recording during performance of 4 tasks selected from shoulder-elbow FMA and 24 patients were involved in the study. Accelerometer-based estimation was obtained by preprocessing raw sensor data, extracting data features, selecting features based on Genetic Algorithm and ELM. Then 4 single-task models and a comprehensive model were built individually using the selected features. Results show that it is possible to achieve accurate estimation of shoulder-elbow FMA score from the analysis of accelerometer sensor data with a root mean squared prediction error value of 2.1849 points. This approach breaks through the subjective and time-consuming property of traditional outcome measures which rely on clinicians at hand and can be easily utilized in the home settings.